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1, INTRODUCTION 

Pineapple is a tropical plant from Bromeliaceae family and one of the most consumption fruits 
around the world after banana and citrus. The government under East Coast Economics Region (ECER) 
program has allocated 7400 hectares land for pineapple cultivation in order to increase the production of 
pineapple [1]. Pineapple grows under swarm climate, optimum water and no infestation. However, it takes 
about 24 to 39 months to yield fruit before it can be harvested. Moreover, to identify the harvesting time 
become a quite challenging. This is because the quality and sweetness of the pineapple are dependent on the 
time of harvesting, if too early will cause pineapple to become less sweet, meanwhile, harvest it too late will 
cause pineapple to become too juicy [2]. Therefore, pineapple must be harvested before it become 
overmatured. 

Currently, machine vision system can provide a huge advantage in helping farmers to identify the 
fruits’ maturity and the optimal time for harvesting. However, the main problem that limits the capability of 
machine vision system is image recognition process and object detection especially in detecting the pineapple 
image. There are many researches have been conducted to address this problem. Mohammad et al. [2] used 
an image processing technique to categorize N36 pineapple. By using binary ellipse mask together with 
morphological normalized RGB to filter out the background, the research showed a promosing result. 
However, the used of binary ellipse mask has its own drawback where the segmented region did not include 
the entire region of interest. The minimum symmetrical distance method has been proposed by [3] to 
overcome the problem mentioned in above. The author managed to segment the entire region of interest from 
the background. However, the proposed method only works using symmetrical shape pineapple only. 
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Kamaruddin et al. [4] have chosen Otsu’s Method as their image segmentation technique before 
applying morphological process. Otsu’s Method assuming the image contains two classes of pixels following 
bi-modal histogram (foreground and background) and it calculates the optimum threshold separating the two 
classes. Prabha and Kumar [5] also used threshold method to segment the banana from the black background. 
However, threshold method only works efficiently if the image contains two colors only. So, the accuracy 
will gradually decrease if the image contains more than two colors. Then the used of g-r gray image was 
proposed by [6] to remove the background from the image. 

Based on the literature review, it was found that to extract fruit from background at the plantation 
level is very difficult and a few factors need to be considered. Firstly, the algorithm must be robust for both 
immature and mature fruits. Thus, most of the researchers only focusing on mature fruit because the intensity 
between mature fruit and background are easily to distinguished. On the other hand, the intensity between 
immature fruit and background is almost the same. Therefore, to apply the algorithm on immature fruit 
cannot be used without modifying the parameters. Secondly, the algorithm also must be robust against 
natural illumination. Thirdly, the fruit in plantation mostly cluster together or occlusion with other fruits, 
leaves or branches. Therefore, to improve the robustness of the algorithm, researchers have combined the 
process of color segmentation with other techniques such as feature extraction, edges detection, blob-based 
segmentation and others [7]-[10]. 


2. RESEARCH METHOD 

Image segmentation is a very important process to remove the unwanted background from the main 
object in one image. In this case is pineapple fruit, in order to ensure that the output image has high accuracy. 
The process starts by the data acquisition, all the images going to Cascade Object Detector (COD) to confirm 
the fruit’s location before going to Hue Value Segmentation (HVS) process. The overall procedure for this 
process is shown in Figure 1. Therefore, to make sure the unwanted background is successfully removed 
from the image, this work has developed a Convolutional Neural Network (CNN) as a searching engine to 
evaluate the results from Adaptive Red and Blue Chromatic Map (ARB) and Normalized Difference Index 
(NDI. 
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Figure 1. The overall process 


2.1. Data Acquisition 

Data aquisition for image has been captured during day time with various illumination conditions at 
pineapple plantation at Pekan Nanas, Pontian, Johor. The captured images have been processed and finally all 
these datas are used to develop an algorithm for pineapple image segmentation technique. In this work, MD2 
type pineapple is selected based on the suggestion from Malaysia Pineapple Industry Board (MPIB), which 
one of the most popular and highest demand in Malaysia. Figure 2 shows the sample images used in this 
project. 


Comprehensive Pineapple Segmentation Techniques with Intelligent ... (Muhammad Azmi Ahmed Nawawi) 


1100 O ISSN: 2502-4752 





Figure 2. Sample images with different stage of maturity and background 


2.2. Algorithm 

Figure 3 shows the flowchart of the proposed algorithm for segmentation process and detecting the 
pineapple object. Since the image used in this project consists of pineapple and background, image 
segmentation process is implemented to remove any unwanted background from the focus image. 
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Figure 3. Flowchart for the proposed algorithm 
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The proposed algorithm starts with data acquisition, by acquiring image from the camera. Since the 
size of the captured image is too large, this project has conducted a few tests to find out suitable scale to 
resize the dimension of the image. By reducing the dimension of the image, the computational time has been 
reduced. After that, the proposed algorithm begins to remove the background in the image by doing image 
segmentation. Firstly, Cascade Object Detector (COD) is implemented to detect the location of the pineapple 
in the image. By returning the bounding box around the detecting pineapple, the proposed algorithm will 
remove all image (pixel) outside this bounding box. By doing this, the algorithm can further reduce the 
computational time and increasing the accuracy for the next process. 

Secondly, the Hue in Hue, Saturation and Value (HSV) color space is extracted and by setting 
threshold value obtained from the experimental test to detect ground and sky. Since the further image 
segmentation is quite difficult, it is necessary to remove any background that is easily to distinguished from 
the fruit. The Hue value for ground (orange) and sky (white) are easily differentiated from other colors. The 
threshold value has been selected based on these properties. Any value less than threshold value is removed 
(pixel intensity becomes 0), meanwhile, the value greater or equal than threshold value is maintained. In 
image segmentation, there are often problem where some images only work well using certain method and 
other images using different method. Therefore, if the same image is used from different method, the interest 
object cannot be segmented completely from the background and vice verse. Thus, this will be resulting in 
low accuracy algorithm if the multiple images are being used. Therefore, to overcome this problem, Adaptive 
Red and Blue Chromatic Map (ARB) and Normalized Difference Index (NDI) at the same time have been 
implemented. 

The threshold value obtained from the experimental test is used in ARB to segment the potential 
pineapple fruit from the leaves. Similar to the HVS, any ARB value which less than threshold value is 
removed (pixel intensity becomes 0) from the image. This value is considered as leaves. On the other hand, 
ARB value for fruit is greater or equal than threshold value. After implement ARB, the result still shows the 
misclassified pixels and the edge of the detecting pineapple is not smooth. This is because ARB value for 
background especially for darker leaves having almost the same value with the fruit. Therefore, to remove 
misclassified error and smooth the edge of the detecting pineapple, this project has proposed the use of 
Template Matching Method (TMM) for further segmentation process. The major axis and minor axis are 
calculated based on the resulting result from the ARB to find the right template, which then is used to remove 
any misclassified pixels and smooth the edge of the detecting pineapple [7]. 

The implementation of ARB and NDI will produce two segmentations image, therefore, the 
Convolutional Neural Network (CNN) is implemented in this algorithm as decision maker. The purpose of 
implementation of CNN is to select the best segmentation image from ARB and NDI. Firstly, the CNN will 
determine the segmentation image produced from ARB and NDI whether the object is a pineapple or not. If 
the segmentation image is a pineapple, the algorithm will accept the segmentation image. Meanwhile, if the 
segmentation image is not a pineapple, the algorithm will reject it. However, if both segmentation images are 
classified as a pineapple, the CNN will go to the second step by using the weight produced when classify the 
image to determine the highest weight between ARB and NDI. Finally, the system will accept only the output 
segmentation image with the highest weight. 


3. RESULTS AND ANALYSIS 

This section presents the result and the analysis of the image accuracy from various segmentation 
stages such as Cascade Object Detector (COD) to detect location of the pineapple, Hue Value Segmentation 
(HVS) to remove ground and sky, Adaptive Red and Blue Chromatic Map (ARB) and Normalized Difference 
Index (NDI) are for leaves removal, Template Matching Method (TMM) for segmentation enhancement and 
edge smoothing and Convolutional Neural Network (CNN) have been used as decision maker to select the 
best segmentation image. 


3.1. Cascade Object Detector (COD) 

Firstly, a Cascade Object Detector (COD) has been selected to detect the location of the pineapple 
from an captured image. Refer to Figure 4, the COD returns the bounding box around the detecting pineapple 
and remove all pixels outside this bounding box. By implemented COD, pineapple can be located anywhere 
in the image and this method has increased the accuracy as shown in Figures 4(a) and 4(b) because the 
background of the selected image has been decreased. 
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(a) (b) 
Figure 4. Result after applying COD (left=original image, right=result after 
COD) 


3.2. Hue Value Segmentation (HVS) 

The second step is the selection of Hue parameter. The accuracy will increased due to the 
background in the image has been decreased. Refer to Figure 5, the ground has been successfully removed 
from the image and other objects are maintained. Finally, it has been found that the output image still can be 
considered as acceptable, even though a few interest pixels have been removed. 





(a) (b) 
Figure 5. Result after applying HVS (left=result after COD, right=result after 
HVS) 


3.3. Adaptive Red and Blue Chromatic Map (ARB) and Normalized Difference Index (NDI 

The third step is to remove leaves from the image as shown in Figure 6 using ARB and NDI 
methods. It can be seen that, the ARB and NDI have successfully removed the leaves from the image as 
shown in Figures 6(b) and 6(d) respectively, while maintained the fruit area and shape. In addition, it has 
been found that the output images in Figures 6(b) and 6(d) still can be considered as acceptable, even though 
a few interest pixels have been removed. However, when using image in Figure 6(a), the ARB has considered 
all pixels in the image as background. This is because the ARB’s value for the all pixels in the image is less 
than threshold values. However from NDI method, the result is much better as shown in Figure 6(c). 
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(b) 





(c) (d) 
Figure 6. (a), (b) Result after applying ARB. (c), (d) Result after applying NDI 


3.4. Template Matching Method (TMM) 

The result from ARB and NDI segmentation methods have a few misclassified pixels and the edge 
of the detecting pineapple is not smooth. Therefore, this project applies additinal image enhancement process 
for further segmentation using Template Matching Method. The different in shape and size of template have 
been constructed in advanced and store in the database. The right template is selected based on the major axis 
and minor axis. Then, the pineapple shape is constructed to produce a smooth edge and all the pixels outside 
this shape are eliminated as shown in Figure 7. 





(a) (b) 
Figure 7. Result after applying template matching (a) Immature pineapple (b) Mature 
pineapple 


3.5. Convolutional Neural Network (CNN) 

The implementations of ARB and NDI have produced two segmentation images. Refer to Figure 8, 
Convolutional Neural Network (CNN) has successfully selected the best segmentation image between ARB 
and NDI. The implementation of CNN in the algorithm has increased the efficiency and accuracy by 
allowing the algorithm to have many methods to detect pineapple. 


Comprehensive Pineapple Segmentation Techniques with Intelligent ... (Muhammad Azmi Ahmed Nawawi) 


1104. O ISSN: 2502-4752 





(a) (b) (Cc) 
Figure 8. Result after applying CNN (a) Result from ARB (b) 
Result from NDI (c) Result from CNN 


4. CONCLUSION 

The image segmentation and detection algorithms have been presented to detect pineapple fruit 
based on color maturity indices. The results found show the used of Cascade Object Detector (COD) was 
very effective and has good potential in detecting the location of the pineapple. The use of Hue value with 
Adaptive Red and Blue Chromatic Map (ARB) and Normalized Difference Index (NDI) were very effective 
for background removal process from the focus image. The implementation of Template Matching Method 
(TMM) has improved the accuracy of the algorithm by detecting the shape of the pineapple and then 
eliminated other pixels with a smooth edge. Furthermore, the implementation of Convolutional Neural 
Network (CNN) as decision maker has increased the efficiency of the algorithm and at the same time 
increased the accuracy in segmenting the fruit from the background. Further research needs to be done using 
intelligent optimization technique for optimal maturity identification. 
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